An Efficient Knowledge Transfer Solution to a Novel SMDP Formalization of a Broker's Decision Problem

Retail and wholesale broker’s decision problems have been optimized separately, ignoring the probable existence of a globally optimal trading strategy. To address this, we propose a novel formalization, based on a semi-Markov decision process (SMDP) and solved using hierarchical reinforcement learning (HRL) in multi-agent environments. Furthermore, to mitigate the curse of dimensionality, which arises when applying SMDP and HRL to complex decision problems, we propose an efficient knowledge transfer approach. An analysis of our controlled experiments in two well-established multi-agent simulation environments within the Trading Agent Competition (TAC) community shows that this broker can outperform the top TAC-brokers and is able to reuse the trading knowledge acquired in previously experienced settings.